Supporting Application Integrated Load Distribution for Message Passing Programs

Message passing libraries for parallel programming like PVM [SGDM94] and the emerging MPI standard [Wal94] are more and more used as programming environments for parallel applications in workstation clusters or distributed memory supercomputers. Simple usage of the programming environment, flexibility of the programming model, and efficiency of implementations are main aspects of the ongoing discussion about these libraries. This paper emphasizes an aspect which is often neglected by programming environments but which is essential for an efficient execution: the support of load distribution. Actual systems supporting load distribution (for example, Dynamo [Tär94]) manage load distribution by directing the assignment of newly generated subproblems to processes of the parallel program (non-preemptive load distribution). This so called task farming or pool of tasks paradigm is only useful for a relatively small number of applications. Most numerical algorithms and simulation programs cannot be efficiently realized based on the task farming paradigm: They need a preemptive mechanism for migrating subproblems to other processes. This paper presents a new portable library for the support of non-preemptive and preemptive load distribution, called ALDY (Adaptive Load Distribution System) [Sch95]. Section 2 describes basic concepts and shows the ALDY user interface. Section 3 illustrates an ALDY application. Section 4 presents experimental results using ALDY for a PVM program on a local area network. A summary and some remarks on our future work are given in section 5.